COVID-19 Detection via Fusion of Modulation Spectrum and Linear Prediction Speech Features

The coronavirus disease 2019 (COVID-19) pandemic has drastically impacted life around the globe. As life returns to pre-pandemic routines, COVID-19 testing has become a key component, assuring that travellers and citizens are free from the disease. Conventional tests can be expensive, time-consuming (results can take up to 48h), and require laboratory testing. Rapid antigen testing, in turn, can generate results within 15-30 minutes and can be done at home, but research shows they achieve very poor sensitivity rates. In this paper, we propose an alternative test based on speech signals recorded at home with a portable device. It has been well-documented that the virus affects many of the speech production systems (e.g., lungs, larynx, and articulators). As such, we propose the use of new modulation spectral features and linear prediction analysis to characterize these changes and design a two-stage COVID-19 prediction system by fusing the proposed features. Experiments with three COVID-19 speech datasets (CSS, DiCOVA2, and Cambridge subset) show that the two-stage feature fusion system outperforms the benchmark systems of CSS and Cambridge datasets while maintaining lower complexity compared to DL-based systems. Furthermore, the two-stage system demonstrates higher generalizability to unseen conditions in a cross-dataset testing evaluation scheme. The generalizability and interpretability of our proposed system demonstrate the potential for accessible, low-cost, at-home COVID-19 testing. IEEE

How Generalizable and Interpretable are Speech-Based COVID-19 Detection Systems?: A Comparative Analysis and New System Proposal

Recent work has shown the potential of using speech signals for remote detection of coronavirus disease 2019 (COVID-19). Due to the limited amount of available data, however, existing systems have been typically evaluated within the same dataset. Hence, it is not clear whether systems can be generalized to unseen speech signals and if they indeed capture COVID-19 acoustic biomarkers or only dataset-specific nuances. In this paper, we start by evaluating the robustness of systems proposed in the literature, including two based on hand-crafted features and two on deep neural network architectures. In particular, these systems are tested across two international COVID-19 detection challenge datasets (COMPARE and DICOVA2). Experiments show that the performance of the explored systems degraded to chance levels when tested on unseen data, especially those based on deep neural networks. To increase the generalizability of existing systems, we propose a new set of acoustic biomarkers based on speech modulation spectrograms. The new biomarkers, when used to train a simple linear classifier, showed substantial improvements in cross-dataset testing performance. Further interpretation of the biomarkers provides a better understanding of the acoustic properties of COVID-19 speech. The generalizability and interpretability of the selected biomarkers allow for the development of a more reliable and lower-cost COVID-19 detection system.

Instrumenting a virtual reality headset for at-home gamer experience monitoring and behavioural assessment

Measuring a gamer’s behaviour and perceived gaming experience in real-time can be crucial not only to assess game usability, but to also adjust the game play and content in real-time to maximize the experience per user. For this purpose, affective and physiological monitoring tools (e.g., wearables) have been used to monitor human influential factors (HIFs) related to quality of experience (QoE). Representative factors may include the gamer’s level of engagement, stress, as well as sense of presence and immersion, to name a few. However, one of the major challenges the community faces today is being able to accurately transfer the results obtained in controlled laboratory settings to uncontrolled everyday settings, such as the gamer’s home. In this paper, we describe an instrumented virtual reality (VR) headset, which directly embeds a number of dry ExG sensors (electroencephalography, EEG;electrocardiography, ECG;and electrooculography, EOG) to allow for gamer behaviour assessment in real-time. A protocol was developed to deliver kits (including the instrumented headset and controllers, laptop with the VR game Half-life Alyx, and a second laptop for data acquisition) to participants’ homes during the COVID-19 lockdown. A brief videoconference session was made to provide the participants with instructions, but otherwise the experiment proceeded with minimal experimenter intervention. Eight participants consented to participate and each played the game for roughly 1.5 h. After each gaming session, participants reported their overall experience with an online questionnaire covering aspects of emotions, engagement, immersion, sense of presence, motion sickness, flow, skill, technology adoption, judgement and usability. Here, we describe our obtained findings, as well as report correlations between the subjective ratings and several QoE-related HIFs measured directly from the instrumented headset. Promising results are reported. Copyright © 2022 Moinnereau, Oliveira and Falk.

FUSION OF MODULATION SPECTRAL AND SPECTRAL FEATURES WITH SYMPTOM METADATA FOR IMPROVED SPEECH-BASED COVID-19 DETECTION

Existing speech-based coronavirus disease 2019 (COVID-19) detection systems provide poor interpretability and limited robustness to unseen data conditions. In this paper, we propose a system to overcome these limitations. In particular, we propose to fuse two different feature modalities with patient metadata in order to capture different properties of the disease. The first feature set is based on modulation spectral properties of speech. The second comprises spectral shape/descriptor features recently used for COVID-19 detection. Lastly, we fuse patient metadata in order to improve robustness and interpretability. Experiments are performed on the 2021 INTERSPEECH COVID Speech Sub-Challenge dataset with several different data partitioning paradigms. Results show the importance of the modulation spectral features. Metadata, in turn, did not perform very well when used alone but provided invaluable insights when fused with the other features. Overall, a system relying on the fusion of all three modalities showed to be robust to unseen conditions and to rely on interpretable features. The simplicity of the model suggests that it can be deployed in portable devices, hence providing accessible COVID-19 diagnostics worldwide. © 2022 IEEE

Context-Aware Speech Stress Detection in Hospital Workers Using Bi-LSTM Classifiers

Hospital workers are known to work long hours in a highly stressful environment. The COVID-19 pandemic has increased this burden multi-fold. Pre-COVID statistics already showed that one in every three nurses reported burnout, thus affecting patient satisfaction and the quality of their provided service. Real-time monitoring of burnout, and other underlying factors, such as stress, could provide feedback not only to the clinical staff, but also to hospital administrators, thus allowing for supportive measures to be taken early. In this paper, we present a context-aware speech-based system for stress detection. We consider data from 144 hospital workers who were monitored during their daily shifts over a 10-week period;subjective stress readings were collected daily. Wearable devices measured speech features and physiological readings, such as heart rate. Environment sensors, in turn, were used to track staff movement within the hospital. Here, we show the importance of context-awareness for stress level detection based on a bidirectional LSTM deep neural network. In particular, we show the importance of hospital location and circadian rhythm based contextual cues for stress prediction. Overall, we show improvements as high as 14% in F1 scores once context is incorporated, relative to using the speech features alone.